Skip to content

feat: Update HuggingFaceLocalChatGenerator default model to Qwen/Qwen3-0.6B #10176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vblagoje merged 12 commits into from
Dec 11, 2025
Merged

feat: Update HuggingFaceLocalChatGenerator default model to Qwen/Qwen3-0.6B #10176

vblagoje merged 12 commits into from
Dec 11, 2025
+52 −11

Conversation

@vblagoje
Copy link
Member

@vblagoje vblagoje commented Dec 2, 2025 •
edited
Loading

Why

Updates HuggingFaceLocalChatGenerator to use Qwen/Qwen3-0.6B as a default model and adds support for thinking-capable models that can generate intermediate reasoning steps.

What

  • Updated default model from generic to Qwen/Qwen3-0.6B
  • Added enable_thinking parameter to enable intermediate reasoning in thinking-capable models
  • Added live integration tests for both thinking modes (enabled/disabled)

How can it be used

# Enable thinking mode for reasoning tasks
llm = HuggingFaceLocalChatGenerator(
    model="Qwen/Qwen3-0.6B",
    enable_thinking=True
)

How did you test it

  • Added live integration tests validating thinking mode behavior with and without enable_thinking
  • Full test suite passes

Notes for the reviewer

  • Default model change may affect existing pipelines relying on the previous default
  • Thinking mode only works with compatible models like Qwen3 and other new models

@vercel
Copy link

vercel bot commented Dec 2, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
haystack-docs Ignored Ignored Preview Dec 11, 2025 10:27am

@vblagoje vblagoje added ignore-for-release-notes PRs with this flag won't be included in the release notes. and removed topic:tests labels Dec 2, 2025
@github-actions github-actions bot added the type:documentation Improvements on the docs label Dec 2, 2025
@vblagoje vblagoje marked this pull request as ready for review December 10, 2025 13:56
@vblagoje vblagoje requested a review from a team as a code owner December 10, 2025 13:56
@vblagoje vblagoje requested review from anakin87 and sjrl and removed request for a team December 10, 2025 13:56
@vblagoje
Copy link
Member Author

vblagoje commented Dec 10, 2025

@sjrl have @anakin87 take a look - he knows this code inside out

anakin87
anakin87 requested changes Dec 10, 2025
View reviewed changes
vblagoje and others added 3 commits December 11, 2025 11:00
@vblagoje @anakin87
Update haystack/components/generators/chat/hugging_face_local.py
530a35f 
Co-authored-by: Stefano Fiorucci <[email protected]>
@vblagoje
Update release notes for HuggingFaceLocalChatGenerator
5e00220 
Updated the release notes to reflect changes in the HuggingFaceLocalChatGenerator, including the new default model and the addition of the enable_thinking parameter.
@vblagoje
Simplify test_live_run with/out enable_thinking flag
936a149
anakin87
anakin87 reviewed Dec 11, 2025
View reviewed changes
@vblagoje vblagoje requested a review from anakin87 December 11, 2025 10:38
@vblagoje vblagoje merged commit 0c214c1 into main Dec 11, 2025
23 checks passed
@vblagoje vblagoje deleted the hf_update branch December 11, 2025 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sjrl sjrl sjrl approved these changes

@anakin87 anakin87 anakin87 approved these changes

Assignees

No one assigned

Labels

ignore-for-release-notes PRs with this flag won't be included in the release notes. topic:tests type:documentation Improvements on the docs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@vblagoje @sjrl @anakin87